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P_] ■ Artificial neural networks built from two-state neurons are powerful computa- 

tional substrates, whose computational ability is well understood by analogy with 
statistical mechanics. In this work, we introduce similar analogies in the context 
of spiking neurons in a fixed time window, where excitatory and inhibitory inputs 
drawn from a Poisson distribution play the role of temperature. For single neurons 
with a "bandgap" between their inputs and the spike threshold, this temperature 
allows for stochastic spiking. By imposing a global inhibitory rhythm over the 
fixed time windows, we connect neurons into a network that exhibits synchronous, 
clock-like updating akin to neural networks. We implement a single-layer Boltz- 
mann machine without learning to demonstrate our model. 
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How networks of spiking neurons compute in the presence of noise is a topic of active re- 
search |fT5l l4l [TT1 . One intriguing possibility is that the noise plays an important role, similar to how 
stochasticity in artificial neural networks is used to solve a wide-range of problems, from recogni- 
tion to optimization (6). However, despite initial progress J7), there is still no systematic way to 
implement stochastic algorithms commonly found in neural networks using spiking neurons. 
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The fundamental problem is that the mathematical framework of artificial neural networks, which 
has deep ties to statistical mechanics, lacks a meaningful definition of temperature in the presence 
of dynamically spiking neurons. For example, the dynamics of the well-known Hopfield network 
are governed by an energy function that converges to a set of stable attractors [ 10 1. However, when 
two-state neurons in the Hopfield network are replaced by stochastically spiking neurons, the net- 
work exhibits non-equilibrium dynamics. In other words, spiking networks do not reach a thermal 
equilibrium, and hence the link between network dynamics and statistical mechanics is broken Q. 

In this paper, we describe a methodology for mapping artificial neural networks onto a network of 
stochastic spiking neurons. In particular, we present an architecture where Poisson input noise sets 
the thermodynamic "temperature" of the network. Our approach is twofold: First, we define the 
notions of energy and temperature for an integrate-and-fire neuron. The insight here is to set a fixed 
time window for the neuron to spike, analogous to a discrete time step in an artificial neural network. 
Second, we construct a network and impose a global inhibitory rhythm that restricts each neuron to 
spike within a synchronized time window, thereby ensuring measures of energy and temperature are 
consistent across the network. Based on this algorithm, it is possible to implement a range of neural 
network algorithms (e.g., the Boltzmann machine [9|) built from neurons that spike. 
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2 Thermodynamic equivalents for spiking neurons 



To arrive at thermodynamic measures in the context of spikes, our strategy is to find a regime where 
the behavior of a two state neuron from artificial neural networks can be approximated by a spiking 
neuron. 

2.1 Bandgap of a I&F neuron 

In artificial neural networks, the state of an abstract two-state neuron, Si, with an input x and 
threshold x t is set by 

f 1 for x > x t , 
S ' 1 ~ 1 for x < x t 

for each discrete time step. We refer to the above neuron model as a "two-state" neuron. For 
subthreshold input, the "bandgap" (e g ) is defined as the additional input required to switch states, 
given by e g = x t — x. 

To introduce the notion of a bandgap for a neuron that spikes, consider the dynamics of a leaky 
integrate-and-fire model (I&F) 

T m u(t) = u a - u(t) + R(I in (t) - I a (t)), 

where r m is the membrane time constant, u{t) is the membrane potential, /,„ (t) is the input current, 
I a (t) is a spike adaptation current, and R is the input resistance. For each instance that u exceeds a 
threshold u t , the neuron generates a spike denoted by times v>f \ after which the membrane is reset 
to u a . Each spike also causes the spike adaptation current to increase as a (^-function impulse with 
magnitude A Q , and decays according to 

rJ a (t) = -I a (t) + A a 5(t-t^), 
where r a is the adaptation time constant. 

This spiking model mimics the behavior of a two-state neuron under the following constraints: 

1 . The opportunity for spiking is restricted to a fixed time window Tw where u has the initial 
condition u Q . 

2. The neuron's membrane time constant is short compared to the fixed time window (r m <C 
Tw)- 

3. The output of the neuron is cast into a binary variable ("on" or "off), defined as the event 
that the neuron spikes at least once in the window. 

4. The input is the fixed value I Q within Tw- 

Applying these constraints, the I&F neuron elicits a spike ("on" state) or not ("off state) based 
on the level of input current I Q . In Figure QJ), the different traces represent the response of the 
membrane potential to different inputs, applied in independent trials. The transfer function, which 
links the neuron's state to its input strength, is identical to that of a two-state neuron, where a critical 
value of input is required to spike (Figure QJ). 

By analogy to the two-state neuron, we denote the bandgap of the I&F neuron as the additional input 
required to spike in Tw, which corresponds to e g ~ Ut — RI a - Note that because the exponential 
trajectory saturates quickly to RI Q (after 3r m ), the instantaneous distance to threshold is approxi- 
mately fixed (see FigureQJ)). Next, we demonstrate that adding noisy inputs to the I&F neuron leads 
to stochastic spiking — analogous to how adding a non-zero temperature to a two-state neuron leads 
to stochastic flipping of states. 
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Figure 1 : I&F as a two-state neuron 
(a) A point l&F neuron that receives direct input current I a , (b) Membrane potential response for 
different levels of I ranging from 0.6 to 2. In this simulation, r m = 2 ms, R = 1, T w = 15 ms, and 
u t =1 . The membrane follows the exponential trajectory: u(t) = RI (l - e^^) that asymptotes to 
Rio when subthreshold; if the membrane crosses threshold (dotted line), the potential jumps to 2u t 
(spikes), at which point it is reset back to 0; note that in these simulations, our model also includes 
spike adaptation to prevent the neuron from spiking again within the window, (c) The neuron's transfer 
function is a step with a sharp transition at 1 . 



2.2 Non-zero temperature by including Poisson spikes 

To add a non-zero temperature to a two-state neuron, a standard method is to flip its state with a 
probability according to the logistic function 

1 

1 + e t 

where e g is the bandgap and T is the temperature Q. We are able to capture this same basic relation- 
ship in an I&F neuron by driving it — in addition to the constant drive that it already receives — with 
spikes drawn from Poisson statistics. The intuition is that a neuron which would otherwise not spike 
in the fixed time window now has the chance to cross threshold due to the stochastic inputs. The 
odds of stochastically crossing the threshold increase as the bandgap decreases. 

Consider the stochastic arrival of excitatory and inhibitory inputs that impart fixed charges w e and 
Wi respectively, as (5-function impulses with Poisson rates Xe and A/. The input current is modified 
to 

i in (t) = i a + We J2 - - w * E *(* - 

j k 

with the arrival times tj E ^ and determined stochastically from their respective Poisson distribu- 
tions. 

We simulated the dynamics of an I&F neuron with noise for three Poisson rates (in Hz): Xe =920 
and Aj =0 (low-noise regime); A# =5,000 and A/ =6,150 (mid-noise regime); and A# =14,000 
and Xj =17,500 (high-noise regime)Q Sample paths for u{t) are shown for the three noise settings in 
Figure[Z|3 when I a — 0.5 for a fixed threshold (black, blue, and red, respectively); for reference, we 
also show the response with no noise (thick green). As expected, the membrane potential variance, 
and hence the odds of crossing threshold, increases with the level of noise. The neuron's transfer 
function is well fit by the logistic function where the slope of the sigmoid becomes less steep at 
higher noise levels (Figure [2}:), and the corresponding fits for the effective temperature are 0.06, 
0. 15, and 0.24 for low, mid, and high noise respectively. 



'These rates represent the combined effect of N synaptic inputs times the average rate of synaptic events. 
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Figure 2: I&F neuron with noise 
(a) l&F neuron that receives a constant input current I a , along with noisy spike trains from excitatory 
and inhibitory populations with mean rates \ E and A/, respectively, (b) Sample paths of l&F mem- 
brane potential when w e = Wi = 0.1, and I a = 0.5 for: no noise (thick green), low noise (black), mid 
noise (blue), and high noise (red), (c) Measured neuron transfer function for the same three levels 
of noise. The effective temperature is extracted by fitting the measured sigmoid (points) to a logistic 
function (lines). 



2.3 Formula for temperature 

We derive an analytic approximation for temperature in this model as follows: The first assumption 
is that the neuron can only spike after jT m , the region where w t — RI a is approximately constant. We 
choose 7 =3 by inspection^ Next we assume that u follows an Ornstein-Uhlenbeck process with 
mean membrane potential /i = RIo+T m (X e w e — AjiOi) and noise amplitude a 2 = T m {\ e w 2 +\iW 2 ); 
in the regime where spike magnitudes w e and Wi are small and their respective arrival rates are 
large, this is a good approximation (5). Finally, we approximate the distribution of first-passage 
times to cross the threshold, which in general is a difficult distribution to find analytically, with an 
exponential probability density function that depends only on the mean first passage-time (MFPT). 
This approximation is supported by our empirical analysis (data not shown), and also has theoretical 
support |[T4| . 

Under these assumptions, the probability to spike as a function of fi is given by the cumulative 
distribution of first-passage times 

P sp k{») = 1 -e- T »'/ T ", 



where Tl^ = Tw — 7T m . 7), is the MFPT with the initial condition u a = /i, given by 

T^ = r m / f{x)dx, 
Jo 

where f(x) — yfne x (1 + erf (x)) (see 0). We arrive at an approximation for temperature by 
computing the slope, m, at P sp k = 1/2, given by 

m = - log 2 — — / 

2 T w a \ a 

and equating this slope with the midpoint slope of the logistic function, 1/4T. Solving for 
temperature yields 

T = T w a 



2(log2)V m /(^) 



Using the values for low, mid, and high noise regimes, our temperature approximations (0.04, 0.15, 
and 0.25 respectively) closely match the simulation (see Figure|2j;). 



2 Note that 7 is the only free parameter that we have in our analysis. 
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Figure 3: Changing network temperature dynamically 
A population of 128 neurons arranged from top to bottom receives a 33Hz square-wave inhibitory 
rhythm (Inh, blue trace), which enforces the constraint that spikes only occur in the low inhibition 
phase, and noisy excitatory and inhibitory inputs (not shown). Each spike event is indicated by a 
vertical dash at the neuron's location (y dimension) and its time (x dimension), to create a raster plot. 
The top 64 neurons are driven with an input of 0.9, whereas the bottom 64 are driven with 0.45. At 
255ms (arrow), the level of background noise is changed from low to high, representing a change in 
temperature from 0.06 to 0.24. 



3 Implementing a connected network 

When neurons are allowed to spike beyond the confines of a fixed time window, our measure for 
temperature breaks down. Furthermore, it is not clear how to connect multiple neurons together into 
a network. Here, we address these issues by imposing order on neuron dynamics using a global 
inhibitory rhythm. 

3.1 Clocking neurons with an inhibitory rhythm 

The idea of adding a separate neural control structure that essentially "clocks" network activity, was 
explored by Menschik et. al. in the context of a hippocampal model for recalling memories lfl2l . 
The main insight is that a strong inhibitory rhythm forces spiking neurons to operate as if there are 
discrete time steps. For example, a 33Hz square-wave rhythm applied to an I&F neuron restricts the 
opportunity to spike to a 15 ms window, interspersed by periods of high inhibition when no spikes 
can occur. In essence, one can think of the periods when spiking is allowed as discrete updates when 
the neuron is evaluating its inputs, and the inhibitory periods as reset phases. 

Consider the dynamics of an unconnected network of 128 I&F neurons, where neurons in the top 
half are driven by high input current, and neurons in the bottom half are driven by low input current 
(Figure [3]). In addition to this constant input, each neuron receives a strong inhibitory rhythm to 
clock network dynamics, as well as excitatory and inhibitory spikes with Poisson statistics to set the 
effective temperature. 

In Figure|3]we demonstrate the effect of changing the temperature of this network when it is running 
continuously. For the first 255 ms, the temperature of the network is set to 0.06. Neurons in this first 
epoch spike mostly in a deterministic manner (black dashes). Specifically, at high input (top half), 
most neurons spike in each of the 8 phases of the rhythm, while at low input (bottom half), most 
neurons do not spike. After 255 ms, we abruptly raise the temperature to 0.24 by increasing the 
level of Poisson noise. In this second epoch we observe that neurons respond more stochastically 
(red dashes). On average, the odds of spiking to the high input are slightly lower than before, yet the 
neurons are more sensitive to lower inputs. In a sense, we have increased the dynamic range of the 
neuron simply by increasing its temperature; the price to pay, however, is that neurons now spike 
less deterministically (i.e., they may spike in the absence of input, or conversely, they may not spike 
in its presence). 



LowT(/. e = 920, )., = 0) Higher T( l,= 14000, ).,= 17000) 
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(a) A two-layer feed-forward network at zero temperature that implements an XOR function. Neurons 
N1-N2 are driven by external binary-valued inputs InO and In 1 for 30 ms with indicated weights. N1 
in turn inhibits N3, while N2 excites N3. All neurons receive global rhythmic inhibition (connections 
not shown), (b) Neuron responses for all input combinations, and the resulting postsynaptic currents 
induced at N3 (dashed lines). N1-N2 evaluate their inputs in the low phase of the inhibitory rhythm 
(15 to 30 ms), and their spikes generate a delayed step response at N3 that lasts the full duration of 
the second phase (45 to 60 ms). N3 crosses threshold when N2 was active and N1 was not active in 
the previous phase, thus performing an XOR. 



3.2 Linking network states across time 

To implement a connected network, we need to determine how outputs (spikes) are passed along as 
inputs (currents) from local connections within the network. Our strategy is to specify a synaptic 
waveform, a(t), that allows the dynamics of the spiking network to exactly match the synchronous 
update of an artificial neural network. Unlike the Poisson inputs described earlier, these synaptic 
inputs are generated in a deterministic manner (similar to how the input to a two-state neuron is 
generated deterministically, before applying the logistic function and comparing to a random number 
drawn from a uniform distribution). 

In a fully connected artificial neural network, the input to the jth neuron in time step k is 
written as Xj(k) — Wj P s p (k — 1), with Wj p as the connection strength between neuron j 

and p, and s p (k — 1) is the binary value of neuron p from the previous step. This update rule 
can be mapped to a network of I&F neurons as follows: First, the input to each neuron, 7 D , is 
mapped to x for each discrete time step. Next, the discrete time step k maps to the time period 
TV + 2TV(fc — 1) < t < 2T\yk, corresponding to the fcth low phase of the inhibitory rhythm. 
Then, s p (k) is taken to be the binary event that neuron p spikes in fcth phase. Finally, the post- 
synaptic current at neuron j induced by a presynaptic spike from neuron p is a delayed pulse given by 

a{t) jp = w jp e(t - t p f ) + T w )(l - Q(t - tCO + 37V)), 

where 0(f) is the Heaviside step function. The first step (left term) models a fixed axonal delay of 
TV, and ensures that spikes in the current window do not influence targets in this same window. The 
second step (right term) sets the length of the pulse at 27V, an d ensures that its effect lasts the full 
duration of the following phase. Note that the form of a(t) was chosen to exactly match the neural 
networks case — variations of a(t) are described in the discussion. 

With these choices, one can use spiking neurons to mimic the dynamics of an interconnected network 
of two-state neurons. For a simple demonstration, we map the well known XOR problem to a 
feed-forward network of three I&F neurons (Figure @). In the following section, we show how to 
implement a stochastic recurrent network, known as the Boltzmann machine. 
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4 A one-layer Boltzmann machine 



The Boltzmann machine, introduced by Ackley et. al. in 1985, is a stochastic neural network that has 
been used to perform recognition, as well as learn data representations without supervision (9). For 
simplicity, we implement a fully-connected one-layer Boltzmann machine without learning using 
spiking neurons. However, it is worth noting that the Boltzmann machine has a local learning 
rule that is ripe for implementation using spike-based mechanisms (e.g., spike-timing dependent- 
plasticity). Our primary goal is to demonstrate that the network's temperature, which we set by the 
level of Poisson noise, leads to stochastic flipping between attractor states. 

The Boltzmann machine algorithm that we consider has N-connected two-state neurons 
Si, S2, ■ ■ ■ , sjv such that the state of neuron i flips on with probability 

1 

Pi = — , 

1 + exp(- JZj W H S i/ T ) 

where T is the thermodynamic temperature. As is standard for the Boltzmann machine, we remove 
self-connections (wu — for all i), and require symmetry (aiy = Wji for all i, j). By applying the 
ideas developed in this paper, we are able to map the above algorithm onto a recurrent network of 
spiking neurons (Figure [5^). 

To test the mapping, we simulate a network of 128 recurrently-connected I&F neurons using a 33Hz 
inhibitory rhythm and a temperature of 0.24. The weight matrix wij is constructed according to the 
method used in [ 10 1. Specifically, we choose four random binary attractors, V s , each of length N 
and 75% sparsity, and compute Wy = (2Vj — 1)(2VJ — 1). Note that attractors are chosen to 
overlap with at least one other attractor by at least 30%. 

We explored the attractor dynamics by correlating binary-valued states SV(fc) = {s.^ . . . s^} in cy- 
cle k with the four stored patterns V s by taking the inner product. Indeed, the correlation closely 
matched one of the four patterns for the majority of the simulation (Figure^). On occasion, the 
network abruptly transitions between states due to the network's non-zero temperature — a hallmark 
trait of a Boltzmann machine with discrete attractors. Figure 5c shows the detailed dynamic transi- 
tion between states 1 and 4 within a specified time window. 

5 Discussion 

We presented a recipe for implementing common neural network algorithms using spiking neurons. 
We build on previous work lfl2l by making explicit parallels to statistical mechanics. To our knowl- 
edge, this is the first time a measure of thermodynamic temperature has been linked to neurons that 
spike probabilistically. 

We view this work as a first step to bringing more biological relevance and physicality to abstract 
two-state neurons. However, as currently presented, the details of our model are biologically unnat- 
ural. 

The membrane time constant used in our simulations is relatively short (2 ms) when compared to 
membrane time constants typically observed in the cortex (10 to 20 ms). However, if biological 
neurons are sufficiently leaky, their effective time constant can be quite short (<5 ms) even though 
the membrane time constant is longer Q. 

More fundamentally, the short membrane time constant was chosen to ensure the membrane poten- 
tial saturates quickly, leading to an intuitive definition for the bandgap. For operation outside of the 
T m <C Tw regime, the membrane potential does not saturate and confounds the measurement of 
network temperature. We are able to define an effective bandgap for the window, however, further 
simulations are required to determine under what conditions a logistic function serves as a good 
approximation of the neuron transfer function. However, in the case r m 3> TW (when the time- 
dependent decay within a time step can be ignored), the transfer function is well known and closely 
resembles a logistic fTTll . 

To emulate discrete time steps in our network, we introduced a square wave inhibitory rhythm that 
is strong enough to reset each neuron's state at a regular interval. Such a rhythm is unrealistic for a 
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Figure 5: Boltzmann Machine with I&F neurons 
(a) A recurrent network of 128 l&F neurons that receives: noisy inputs from excitatory and inhibitory 
populations, global rhythmic inhibition, and recurrent inputs, (b) The correlation of the network's 
binary state fluctuates stochastically between four attractors. (c) The detailed network trajectory from 
attractor 1 (red, left) to attractor 4 (green, right), plotted as a spike a raster over multiple inhibitory 
cycles. 



biological network in its present form. For example, it is thought that brain rhythms are sinusoidal, 
and even though spikes can be entrained to the rhythm, they may occur at any phase lfl6ll , Although 
we did not explore these more biologically plausible rhythms, previous work from Menschik et al. 
demonstrates that a moderate-strength sinusoidal rhythm still leads to attractor-like dynamics in a 
recurrent network lfl2l . 



The synaptic waveform used in Section I3.2I mimics a synchronous update in artificial neural net- 
works. This waveform was used only for clarity and is not the only choice. For example, adopting 
a more biologically-relevant synapse with a rise and decay that lasts multiple inhibitory cycles, the 
network state could evolve in a more continuous and asynchronous manner ifTHl . In this case, the 
current state of the network would depend on a handful of previous states each discounted by time, 
and we conjecture that this could provide a "stiffness" against random network fluctuations. 

More pragmatically, this work could have implications for high performance computing. Based 
on the present algorithm, one can implement many off-the-shelf neural network algorithms using 
neuromorphic architectures, which offer a scalable way to build large interconnected networks of 
spiking neurons |fl3ll . Learning the synaptic connections in these large networks then becomes the 
bottleneck. We are currently working to map local learning rules, such as contrastive divergence [8] 
to spiking networks using the same approach presented in this paper. 
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